Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 
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FOREWORD 


Test  and  Evaluation  (T&E)  is  one  of  the  key  enabling  activities  that  underpin  the  ADF’s 
capability  development  process  to  realise  CDF’s  vision  for  a  balanced,  networked  and 
deployable  force.  Defence’s  T&E  approach  will  ensure  that  the  capabilities  that  are  fielded 
are  tested  and  evaluated  and  are  suitable  for  ADF  operations. 

Defence’s  T&E  approach  contributes  to  the  development  of  systems  and  capabilities, 
and  articulates  a  strategy  for  managing  T&E  resources  to  meet  the  needs  of  the  Defence 
Capability  Plan  (DCP). 

The  T&E  Roadmap  achieves  this  by  outlining  the: 

•  factors  that  influence  the  T&E  to  enable  delivery  of  capability; 

•  a  framework  for  understanding  the  T&E  requirements  in  the  future  joint  operating 
environment; 

•  requirement  for  T&E  to  be  conducted  at  all  stages  of  a  capability’s  life  cycle; 

•  Defence’s  current  T&E  capabilities;  and 

•  how  Defence’s  future  T&E  requirements  are  to  be  realised. 

T&E  can  be  employed  to  prove,  demonstrate  or  assess  the  ability  of  proposed  and  existing 
capability  systems,  new  or  upgraded,  to  satisfy  specified  technical  and  operational 
requirements  and  objectives.  When  T&E  is  employed  in  this  manner,  with  the  objective  of 
providing  results  to  inform  decisions  at  key  milestones  in  a  capability  system’s  life  cycle,  it 
becomes  an  effective  component  of  a  capability  risk  management  strategy. 

The  results  of  T&E  will  be  used  to  provide  proof  that  risk  is  contained  within  acceptable 
boundaries  when  making  key  life  cycle  milestone  decisions  and  that  the  intended  system 
meets  both  safety  standards  and  end-users’  requirements. 


This  Roadmap  identifies  Defence’s  key 

actions  for  T&E  as: 

•  development  of  a  centralised  and 
coordinated  approach  to  Defence’s  T&E 
resources  and  priorities; 

•  a  proposed  T&E  framework  that 
considers  T&E  at  the  system,  platform, 
environment  and  joint  capability  levels; 

•  establishment  of  appropriate  and 
sufficient  resourcing  levels  for  T&E  in 
the  development  of  capability;  and 

•  planning  and  accounting  for  an 
increasing  number  of  projects  that 
have  high  technology  combination  of 
complex  subsystems. 


Defence  understands  that  key  partnerships 
need  to  be  established  in  order  to  gain 
the  maximum  leverage  to  assist  with 
the  development  of  its  T&E  capability 
Defence’s  key  partners  include 


•  Industry; 

•  other  Government  agencies;  and 

•  our  allies  and  multi-national  partners. 

Pivotal  to  the  development  of  these 
relationships  is  the  requirement  to  clearly 
articulate  Defence’s  T&E  approach 
and  strategies  to  ensure  the  delivery  of 
appropriate  and  relevant  T&E  to  Defence. 

The  T&E  Roadmap  will  be  revised  to 
remain  synchronised  with  Defence’s 
strategic  direction,  T&E  policy,  capability 
development  processes  and  changes  to  the 
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ART  1 


THE  PURPOSE  OF  DEFENCE  T&E 


T&E  is  a  process  to  obtain  information  to  support  the 
objective  assessment  of  a  Capability  System  with  known 
confidence,  and  to  confirm  whether  or  not  a  risk  is  contained 
within  acceptable  boundaries  across  all  facets  of  a  system’s 
life  cycle. 


CHAPTER  1  -  INTRODUCTION 


PURPOSE 

The  CDF  has  stated  that  his  vision  for  the  ADF  is  a  £ balanced , 
networked,  and  deployable  force,  staffed  by  dedicated  and  professional 
people,  that  operates  within  a  culture  of  adaptability  and  excels  at 
joint,  interagency,  and  coalition  operations .’1  The  aim  of  the  Defence 
Test  and  Evaluation  (T&E)  Roadmap  is  to  review  the  current  Defence 
T&E  capability,  Defence’s  strategic  planning  for  future  operating 
environments,  and  the  future  capability  acquisition  plan  as  detailed  in 
the  DCP  2006-2016,  to  ensure  Defence’s  T&E  direction  and  resources 
will  be  adequate  to  assist  in  the  delivery  of  the  CDF’s  vision. 

This  document  explains  the  benefits  of  T&E  to  Defence  and  how  it  can 
assist  the  capability  development  process.  It  considers  Defence’s 
current  T&E  capability  and  policy,  the  level  at  which  T&E  is  being 
performed,  when  it  is  undertaken,  and  what  T&E  resources  are  utilised. 
Importantly,  the  Roadmap  considers  the  extent  of  the  gaps  and 
overlaps  in  the  delivery  of  T&E  which  may  impact  on  the  delivery  of 
DCP  projects,  and  the  resources  that  will  be  required  to  deliver  these 
projects  into  service. 


1  Joint  Operations  for  the  21st  Century,  page  3 


This  Roadmap’s  broad  aims  include: 

•  reviewing  the  benefits  of  T&E  in  the 
capability  development  process; 

•  providing  an  overview  of  resourcing 
Defence’s  T&E  capability  to  achieve 
successful  project  outcomes  as 
outlined  in  the  DCP; 

•  informing  both  industry  and  the 
Defence  community  of  the  ADF’s  T&E 
strategy;  and 

•  provide  future  T&E  direction  regarding 
identified  shortfalls. 

To  address  these  objectives,  the 

Roadmap  is  divided  into  three  parts 

covering: 

•  the  purpose  of  Defence  T&E; 

•  the  existing  Defence  T&E  environment 
-  current  T&E  capabilities  including 
agencies  and  facilities,  with  extant  T&E 
policy,  principles  and  guidance;  and 

•  the  future  -  the  transformation  of  the 
T&E  environment  to  enable  more  rapid 
fielding  of  capabilities  to  meet  the 
Future  Joint  Operating  Concept  (FJOC). 


T&E  DEFINED 

T&E  can  be  defined  as  a  process  that  obtains  information  to  assist  in  the 
evaluation  or  support  the  objective  assessment  of  a  capability  system  with  known 
confidence,  and  to  confirm  whether  or  not  a  risk  is  contained  within  acceptable 
boundaries  across  all  phases  of  a  system’s  life  cycle. 

PURPOSE  OF  T&E 

The  fundamental  purpose  of  T&E  in  a  Defence  system’s  development  and 
acquisition  program  is  to  identify  the  areas  of  risk  to  be  reduced  or  eliminated. 
During  the  early  phases  of  development,  T&E  is  conducted  to  demonstrate 
the  feasibility  of  conceptual  approaches,  evaluate  design  risk,  identify  design 
alternatives,  compare  and  analyse  trade-offs,  and  establish  satisfaction  of 
operational  requirements. 

As  a  system  undergoes  design  and  development,  the  iterative  process  of  testing 
moves  gradually  from  a  concentration  on  Development  Test  and  Evaluation 
(DT&E),  which  is  concerned  chiefly  with  attainment  of  engineering  design  goals, 
through  Acceptance  Test  and  Evaluation  (AT&E)  to  increasingly  comprehensive 
Operational  Test  and  Evaluation  (OT&E),  which  focuses  on  questions  of 
operational  effectiveness,  suitability  and  survivability. 


Although  there  are  usually  separate 
Development  Test  (DT)  and  Acceptance 
Test  (AT)  events,  DT&E  and  AT&E  are  not 
necessarily  serial  phases  in  the  evolution  of 
system  development.  When  appropriate, 
combined  or  concurrent  DT  and  AT  or 
Operational  Tests  (OT)  are  encouraged. 

This  concurrency  of  testing  can  contribute 
to  savings  in  costs  and  time  in  many 
circumstances. 

Significant  risk  identification  and  mitigation 
is  possible  when  activities  such  as  OT&E 
are  conducted  early  in  a  capability  system’s 
life  cycle. 


CHAPTER  2  -  T&E  IN  THE 
CAPABILITY  LIFE  CYCLE 


CAPABILITY  SYSTEM  LIFE  CYCLE 

Each  capability  system  has  a  ‘life  cycle’  that  begins  with  the 
identification  of  the  need  to  address  a  current  or  prospective  capability 
gap.  This  need  is  progressively  translated  into  a  working  capability 
system  that  is  operated  and  supported  within  one  of  the  combat 
operating  environments  until  it  is  ultimately  withdrawn  from  service. 
The  stages  of  capability  life  cycle  are: 

•  Needs, 

•  Requirements, 

•  Acquisition, 

•  In-Service,  and 

•  Disposal. 

There  are  a  number  of  inputs  which  must  be  considered  when 
developing  future  capabilities  (or  capability  systems)  within  each  of 
the  combat  environments  -  Maritime,  Land,  and  Aerospace  (or  Air 
and  Space),  and  also  within  capability  enablers  such  as  Network 
Centric  Warfare  (NCW).  These  have  been  defined  in  the  Defence 
Capability  Development  Manual  2006  (DCDM)  as  Fundamental  Inputs 
to  Capability  (FIC).  These  inputs  are: 

•  Personnel, 

•  Organisation, 

•  Collective  Training, 

•  Major  Systems, 

•  Supplies, 

•  Facilities, 

•  Support,  and 

•  Command  and  Management. 


T&E  applies  equally  to  all  of  the  above  areas,  not  just  Major  Systems.  T&E 
activities  are  applicable  throughout  the  system’s  life  cycle  and  are  not  confined 
to  system  acceptance  or  transition  to  in-service.  Significant  benefits  will  be 
obtained  through  the  application  of  T&E  activities  early  in  the  capability  system’s 
life  cycle.  The  primary  benefit  is  the  reduction  in  development  costs  through  the 
early  identification  of  risks  and  capability  shortfalls. 


Figure  1  details  the  broad  T&E  undertaken  in  the  life  cycle  of  capabilities. 
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Figure  1 :  Capability  Life  Cycle 


T&E’S  CONTRIBUTION  TO  CAPABILITY 

T&E  contributes  to  ensuring  that  Defence’s  future  capability  needs  are  met  and 
value  for  money  is  achieved. 

The  level  of  T&E  effort  applied  in  support  of  the  development  of  capability  should  be 
proportional  to  the  level  of  risk  identified  at  each  life  cycle  stage  and  in  relation  to 
the  importance  of  the  capability  to  Australia’s  defence  needs. 

Additionally,  T&E  should  be  structured  to  provide  essential  information  to  decision 
makers,  assess  attainment  of  technical  performance  parameters,  and  determine 
whether  systems  are  operationally  effective,  suitable,  sustainable,  safe  and 
environmentally  compliant  for  intended  use. 

The  results  of  experimentation  of  future  concepts  via  Modelling  and  Simulation 
(M&S)  can  be  used  by  T&E  where  appropriate  in  the  Needs  phase.  A  range  of  DT&E 
and  AT&E  activities  will  be  conducted  during  the  Requirements  and  Acquisition 
phases.  OT&E  for  both  Major  and  Minor  projects  mostly  occurs  throughout  the 
In-Service  phase,  although  significant  benefits  can  be  obtained  through  early 
validation  of  a  capability  through  measured  user  input  via  early  OT&E  activities. 

The  conduct  of  T&E  should  facilitate  learning,  assess  technology  maturity 
and  interoperability,  facilitate  integration  into  fielded  capabilities,  and  confirm 
performance  against  documented  capability  needs. 


T&E  AND  THE  SYSTEMS  ENGINEERING  PROCESS 

The  Systems  Engineering  Process  (SEP)  is  the  integrated  set  of  interdisciplinary 
tasks  that  are  required  throughout  a  systems’s  life  cycle,  to  transform  customer 
needs,  requirements,  and  constraints  into  a  system  solution.  The  SEP  focusses 
on  the  total  technical  effort  of  engineering  activities  necessary  to  guide  product 
development  whilst  ensuring  that  the  product  is  properly  designed  to  make  it 
affordable  to  produce,  own,  operate,  maintain,  and  eventually  to  dispose  of, 
without  undue  risk  to  health  or  the  environment.  Furthermore,  the  SEP  includes 
the  life  cycle  processes  for  manufacturing,  test,  distribution,  support,  training, 
and  disposal,  which  are  necessary  to  provide  life  cycle  support  for  products. 

The  T&E  process  is  an  integral  part  of  the  SEP,  which  identifies  levels  of 
performance  and  assists  the  developer  in  correcting  deficiencies.  It  is  a 
significant  element  in  the  decision-making  process,  providing  data  to  support 
trade-off  analysis,  risk  reduction,  and  requirements  refinement.  Program 
decisions  on  system  performance,  maturity  and  readiness  to  advance  to  the 
next  phase  of  development  are  based  on  the  results  of  T&E.  The  issue  of 
paramount  importance  to  the  user  is  system  performance;  that  is,  can  the 
system  fulfil  the  mission?  This  question  applies  to  all  elements  of  FIC,  not  just 
Major  systems.  The  T&E  process  provides  data  that  informs  the  user  on  how 
well  the  system  is  performing  during  development  and  if  it  is  ready  for  fielding. 


Program  managers  are  reponsible  for  balancing  the  risks  of  cost,  schedule,  and 
performance  to  keep  the  program  on  track  through  to  production  and  fielding.  T&E 
in  the  SEP  identifies  risks  and  enables  decision-making  authorities  to  assess  risk 
trade-offs. 

Figure  2  provides  detail  on  T&E  as  part  of  the  SEP. 


Figure  2:  System  Development  V  Diagram 


T&E  results  feature  prominently  in  the  decisions  reached  at  design  and  milestone 
reviews  (decision  points).  However,  the  fact  that  T&E  results  are  required  at  major 
decision  points  does  not  presuppose  that  T&E  results  must  always  be  favourable. 
The  final  decision  responsibility  lies  with  the  decision-maker  who  must  examine 
the  critical  issues  and  weigh  the  facts.  Only  the  decision-maker  can  determine 
the  weight  and  importance  that  is  to  be  attributed  to  a  system’s  capabilities 
and  shortcomings  and  the  degree  of  risk  that  can  be  accepted.  T&E  provides 
the  decision-maker  with  a  solid  base  of  information  upon  which  to  make  these 
judgements. 

The  Cost  Escalation  Model  articulates  that  the  cost  of  correcting  defects  increases 
significantly  the  later  in  the  life  cycle  that  they  are  identified  because  of  increasing 
system  and  sub-system  complexity  and  integration. Because  defects  found  earlier 
in  the  life  cycle  will  cost  significantly  less  to  resolve  than  those  identified  later  in 
the  development  process,  T&E  should  be  applied  early  in  the  system  life  cycle  to 
help  control  system  development  costs. 

Figure  3  shows  the  benefits  of  capturing  defects  early  in  the  life  cycle. 
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Figure  3:  Life  Cycle  Cost  Escalation  Model 

Source:  Defence  Materiel  V&V  Manual 


CHAPTER  3  -  WHY  A  T&E  ROADMAP 


The  T&E  Roadmap  considers  the  current  T&E  policy,  processes, 
organisation,  and  resources  within  Defence,  and  the  future  T&E 
required  to  facilitate  the  delivery  of  capability.  The  T&E  Roadmap  is 
Qj  applicable  to  both  Major  and  Minor  projects. 

STRATEGIC  GUIDANCE 

There  have  been  three  main  reviews  and  reports  that  provide  strategic 
guidance  and  highlight  the  importance  of  T&E  for  Defence.  They  are 
the: 

•  Defence  Procurement  Review  2003; 

•  Auditor  -  General  Audit  Report  No.  30  2001-2002;  and 

•  Report  on  the  Inquiry  into  Materiel  Acquisition  and  Management 
in  Defence  -  Foreign  Affairs,  Defence  and  Trade  References 
Committee,  March  2003. 

Defence  Procurement  Review  2003 

The  Defence  Procurement  Review  2003,  commonly  known  as  the 
Kinnaird  Report,  reviewed  issues  associated  with  major  Defence 
acquisition  projects.  The  report  included  many  recommendations  for  a 


new  approach  to  Defence  acquisition,  and  highlighted  the  need  for  T&E  as  an  essential  tool 
in  the  acquisition  of  Defence  equipment  to  reduce  risk,  define  technical  limits  and  monitor 
contract  performance  and  compliance. Significantly,  the  report  stated  that: 

Greater  resources  need  to  be  allocated  to  conduct  comprehensive  and  rigorous 
T&E  programs  as  part  of  project  funding.  Currently  T&E  is  often  not  conducted 
in  a  comprehensive  and  systematic  way  for  projects,  and  expertise  available  in 
Defence  is  not  fully  utilised.  Although  each  of  the  Services  undertakes  varying 
degrees  of  T&E,  these  activities  need  to  be  integrated  with  the  significant 
expertise  and  experience  available  in  DSTO  through  the  development  of  a  T&E 
concepts  document  during  second  pass.1 

Auditor-General  Audit  Report  No.  30  2001-2002 

The  individual  Defence  Groups  have  formulated  their  own  policies,  practices  and  personnel 
training.  The  Auditor-General  Audit  Report  No.  30  2001-2002  stated  that  Defence 
needs  to  promote  a  ‘unified  approach’  to  T&E  to  guarantee  effective  and  efficient  use  of 
all  T&E  resources,  to  avoid  unnecessary  duplication  of  effort  and  resources.  Accordingly, 
policies  are  being  reviewed  to  implement  how  the  ‘unified  approach’  can  be  achieved.  The 
Roadmap  also  calls  for  T&E  resources  to  be  costed,  to  assist  in  management  and  funding  of 
T&E.  In  practice,  the  cost  of  the  resources  applied  currently  to  T&E  is  generally  inaccurately 
forecasted. 


1  Defence  Procurement  Review  2003,  p.20 


Report  on  the  inquiry  into  materiel  acquisition  and  management  in 
Defence  -  Foreign  Affairs,  Defence  and  Trade  References  Committee, 
March  2003. 

In  Chapter  6  of  its  recommendations,  the  Senate  Committee  addressed  Defence 
T&E  as  follows: 

Successful  project  management  demands  successful  risk 
management.  Weapons  platforms  and  support  systems  must  be 
delivered  to  the  specified  levels  of  functionality  and  with  safety- 
critical  features  assured.  This  can  only  be  achieved  if  test  and 
evaluation  is  given  a  prominent  place  in  the  management  of 
capability  development,  acquisition  and  transition  into  service. 

The  Committee  is  not  completely  satisfied  that  test  and  evaluation 
enjoys  the  status  it  deserves  in  capability  development,  and  notes 
that  a  review  of  T&E  policy  is  currently  underway. 

T&E  policy  has  since  been  amended  by  the  Australian  Defence  Test  and 
Evaluation  Office  (ADTEO). 

Other  Strategic  Guidance 

In  addition  to  these  reviews  and  audits  a  number  of  other  internal  Defence 
documents  provide  relevant  information.  These  include: 

•  DCP  2006-2016; 

•  Defence  Capability  Development  Manual; 

•  Network  Centric  Warfare  (NCW)  Roadmap  2007;  and 

•  other  Defence  T&E  Policies. 


THE  EXISTING  DEFENCE  T&E  ENVIRONMENT 


CHAPTER  4  -  CURRENT  DEFENCE  T&E 


DEFENCE  CAPABILITY  PLAN  2006-2016 

In  the  DCP  2006-2016  there  are  in  excess  of  130  Major  projects.  All 
of  these  projects,  along  with  Minor  and  Rapid  Acquisition  projects, 
require  T&E  of  varying  degrees  to  inform  the  Capability  Manager  whether 
capabilities  are  operationally  effective,  suitable,  sustainable,  safe  for 
intended  use  and  environmentally  compliant. 

$ 

The  DCP,  in  itself,  is  not  a  planning  tool  from  a  T&E  perspective  due  to 
the  nature  of  the  document.  The  DCP  provides  brief  detail  on  the  Major 
Capital  Equipment  (MCE)  proposals  that  are  currently  planned  to  go 
through  the  two-pass  system  during  the  document’s  period.  The  DCP  is 
updated  biannually  and  changes  over  time  as  Defence’s  priorities  change. 
While  the  DCP  provides  a  broad  indication  of  future  capability,  it  does  not 
contain  the  level  of  fidelity  required  for  detailed  T&E  planning  purposes. 
This  level  of  information  is  difficult  to  capture,  but  the  ADTEO  is  now 
developing,  on  behalf  of  CCDG,  a  database  to  provide  a  consolidated 
view  of  strategic  T&E  requirements.  This  will  enable  future  T&E  resource 
planning  and  management.  Figure  4  shows  a  breakdown  of  T&E 
information  on  some  significant  Major  projects  listed  in  the  DCP. 


Figure  4:  DCP  Project  T&E 


The  DCP  indicates  that  there  is  a  strong  upward  trend  in  Defence  procurement 
spending  for  the  next  decade,  which  will  have  considerable  effect  on  T&E 
resourcing,  particularly  for  investment  in  both  T&E  personnel  and  facilities. 

Financial  analysis  anticipates  an  expenditure  of  $6  billion  on  acquisition  during 
the  2011-2012  financial  year.  The  DCP  indicates  the  breakdown  of  acquisition 
expenditure  over  the  period  of  2006-2016  is  as  follows: 

•  46%  on  electronics; 

•  26%  on  aerospace; 

•  1 1  %  on  maritime  ; 

•  10%  on  vehicles  and  land;  and 

•  7%  on  weapons  and  munitions. 

$ 

These  figures  indicate  that  T&E  needs  to  be  reviewed  at  a  Defence  level  to 
ensure  appropriate  levels  of  T&E  resourcing  are  available  to  meet  the  substantial 
increase  in  expenditure.  In  particular,  the  increase  in  expenditure  in  electronics 
-  as  Defence  moves  to  a  network  enabled  force  -  is  an  important  driver  for 
change  in  the  T&E  community. 


T&E  AGENCIES  AND  FACILITIES 

Currently  T&E  leadership,  policy,  management,  coordination  and  guidance  is 
provided  by  the  ADTEO  with  T&E  activities  conducted  by  agencies  at  the  Service/ 
Group  level.  Single  Services  have  a  role  to  play  in  T&E.  Defence  T&E  roles  and 
responsibilities  are  outlined  in  DI(G)  OPS  43-1 ,  Defence  T&E  Policy.  Defence  Trials 
in  accordance  with  DI(G)  ADMIN  06-1  are  conducted  by  ADTEO. 

There  are  a  number  of  other  stakeholders  and  enabling  organisations  within 
Defence  T&E.  These  include  Capability  Development  Group  (CDG),  Defence 
Support  Group  (DSG)  and  Joint  Logistics  Command  (JLC),  within  Vice  Chief  of 
the  Defence  Force  (VCDF)  Group.  Industry  also  has  an  important  place  within  the 
Defence  T&E  community,  through: 

•  contractors/suppliers  that  provide  systems,  conduct  T&E  and  Verification  and 
Validation  (V&V)  as  part  of  the  acquisition  phase;  and 

•  stand-alone  test  agencies  (with  specialist  test  facilities),  that  conduct  tests  and 
trials  on  behalf  of  Defence. 


Many  existing  Defence  T&E  facilities  are  approaching  (or  have  reached)  the 
end  of  their  useful  life  and  without  significant  upgrade  and  investment,  these 
facilities  will  be  unable  to  adequately  meet  future  weapon  system  or  joint 
capability  T&E  requirements.  Some  T&E  activities  cannot  be  conducted  within 
this  country.  The  lack  of  indigenous  missile  ranges  and  targets,  underwater 
testing  and  electronic  warfare  testing  facilities,  or  guaranteed  access  to  such 
overseas  facilities  has  a  major  impact  in  assessing  operational  capability.  Also, 
Defence  cannot  conduct  Electro-Explosive  Hazard  (EEH)  testing  due  to  the  lack 
of  a  suitable  EEH  Test  Facility.  Such  a  capability  would  enable  the  indigenous 
development  of  policy  and  materiel  standards  for  explosive  ordnance. 

Presently  there  is  no  coordinated  approach  at  a  Defence  level  to  identify  T&E 
facilities  necessary  to  support  the  DCP.  Spending  needs  to  be  prioritised  to 
ensure  that  the  appropriate  T&E  facilities  exist  or  are  accessible  via  industry  or 
overseas.  Overall,  T&E  facilities  need  to  be  managed  through  establishment, 
upgrade  and  to  eventual  decommissioning. 

Although  Defence’s  T&E  approach  has  previously  been  platform -based  in  single 
environments,  achieving  the  FJOC  will  necessitate  capabilities  that  routinely 
operate  across  platforms  and  environments.  Accordingly,  Defence  will  need  to 
develop  a  T&E  approach  that  enables  the  application  and  coordination  of  T&E 
across  platforms  and  environments.  Figure  5  depicts  a  model  for  Defence’s 
future  approach  to  T&E. 
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Figure  5:  The  Defence  T&E  Framework 


CHAPTER  5  -  T&E  ISSUES 


A  number  of  current  T&E  issues  and  weaknesses  have  been  identified 
from  previous  studies,  reports  and  through  engagement  with  the 
Defence  T&E  community.  At  the  time  of  writing,  a  number  of  these 
issues  remain  outstanding.  A  summary  of  these  issues  follows. 

Organisational  Issues 

Within  Defence,  there  are  a  number  of  organisations  that  are 
responsible  for  the  management  and  conduct  of  T&E  activities.  In 
relation  to  these  organisations,  the  following  issues  have  been 
identified: 

•  The  extant  organisations  do  not  fully  cover  all  of  the  current  T&E 
requirements  within  the  Capability  Life  Cycle. 

•  T&E  responsibilities  need  to  be  clearly  defined  and  enunciated  so 
as  to  reduce  the  risk  of  nugatory  duplication  of  T&E  management, 
conduct  and  reporting.  An  example  where  this  could  potentially 
occur  would  be  T&E  for  the  MRH-90,  an  Army  led  acquisition  of 

a  platform  which,  once  in  service,  is  to  be  operated  by  both  Army 
and  Navy  in  different  operating  environments. 


While  Maritime  and  Aerospace  T&E  is  generally  well  conducted,  the  Land 
environment  does  not  have  an  organisation  responsible  for  undertaking  OT&E  and 
acceptance  into  service.  At  the  Combat  Environment  and  Joint  Force  Operating 
levels,  Defence  lacks  both  organisations  and  resources  to  manage,  coordinate 
and  undertake  T&E  to  ensure  the  integration  of  capabilities  through  a  common 
methodology  that  is  based  on,  and  aligns  with,  relevant  standards  and  procedures. 

The  lack  of  a  standardised  and  coordinated  approach  to  T&E  within  Defence, 
particularly  across  environments  and  at  a  joint  level,  means  that  it  is  difficult  to 
identify  the  gaps  to  ensure  that  T&E  is  managed  and  conducted  to  an  appropriate 
level  to  identify  risk. 

Industry  Issues 

Information  to  assist  in  the  development  and  conduct  of  T&E  activities  is  often 
difficult  to  obtain;  this  is  often  the  case  when  the  equipment  is  procured  from 
foreign  organisations.  Such  information  is,  nevertheless,  an  essential  contract 
deliverable. 

The  current  approach  to  T&E  during  the  Acquisition  phase  often  appears  to  have 
Industry  performing  T&E  activities  in  isolation  from  the  Commonwealth,  which  may 
adversely  impact  on  the  performance  of  the  final  capability.  Early  collaborative 
involvement  of  Commonwealth  testing  agencies  in  AT&E  activities  will  ensure  the 
T&E  conducted  is  focussed  on  the  critical  operational  issues. 


Resource  Issues 

Financial 

The  Defence  Capability  Development  Manual  (2006)  mandates  the  funding  of 
OT&E  and/or  validation  activities.  However  there  are  constant  pressures  on 
projects  to  under-resource  these  important  activities.  Additionally,  funding  for 
reworking  and  regression  testing  appears  to  be  inadequate  in  many  current 
projects.  It  is  likely  that  this  situation  has  arisen  due  to  inexperience  and 
understanding  of  the  application  and  level  of  T&E  resources  necessary  to 
successfully  support  projects. 

The  lack  of  adequate  T&E  funding  may  mean  that  not  all  deficiencies  are  identified 
through  the  design  and  build  process,  leading  to  potentially  unsafe  systems  or 
shortfalls  in  capability  performance  which  may  lead  to  escalation  in  the  cost  of 
ownership. 

A  lack  of  historical  data  on  the  total  costs  of  T&E  for  a  capability  program  (across 
all  stages  of  the  capability  life  cycle)  makes  it  difficult  for  project  staff  to  estimate 
the  T&E  funding  needed. 

People 

A  number  of  people  issues  were  identified  during  the  development  of  this 
Roadmap,  they  include: 

•  Significant  gaps  in  competency  recognition,  which  potentially  makes  T&E 
less  attractive  than  other  career  paths  that  offer  formal  qualifications  and 
recognition. 


•  No  formalised  career  management  of  personnel  with  T&E  expertise  and 
experience,  which  often  means  that  skills  and  experience  are  ‘lost’  as  staff  move 
into  new  roles  in  non-T&E  organisations. 

•  The  majority  of  T&E  training  is  conducted  ‘on  the  job’  and  staff  turnover 
(especially  with  military  members)  results  in  people  leaving  the  field  before  the 
value  of  their  training  can  be  realised. 

•  Military  members  raised  concerns  that  were  they  to  be  ‘streamed’  into  a 
specialist  T&E  path,  their  opportunities  for  further  promotion  may  be  limited. 

•  The  aging  workforce  means  there  will  be  a  period  in  the  next  5  to  1 0  years 
where  a  number  of  T&E  specialist  personnel  will  retire  from  the  workforce.  A 
lack  of  effective  succession  planning  will  decrease  Defence’s  ability  to  provide 
sufficient  T&E  to  meet  capability  needs.  Currently  80%  of  Defence  Materiel 
Organisation  (DMO)  technical  officers  are  over  40  and  1 7%  are  over  55.  The 
average  age  of  an  DMO  engineer  is  43  with  63%  of  engineers  over  40  and  14% 
over  551. 

•  Difficulty  in  recruiting  and  retaining  Technical  Officers  and  Engineers.  Currently 
in  DMO  Technical  Officer  positions  are  under  filled  by  27%  and  Engineer 
positions  are  under  filled  by  28%1.  Given  the  forecast  skills  shortage  it  is  likely 
that  these  figures  will  increase,  potentially  impacting  Defence’s  capacity  to 
undertake  T&E. 

•  There  is  limited  -  and  in  some  instances,  a  complete  lack  of  -  T&E  expertise  in 
some  specialist  capabilities  within  Defence  or  Australian  industry;  for  example, 
the  skills  to  conduct  large-scale  joint  OT&E  or  to  operate  an  EEH  Test  Facility. 

•  Defence  needs  to  identify  the  skills  and  expertise  to  adequately  undertake  T&E 
in  emerging  technological  environments,  including  the  threat  of  asymmetric 
weapons  and  so-called  ‘disruptive’  technologies. 


1  DMO  Engineering  /  Technical  Job  Family  Workforce  Analysis 


Time 

Traditionally  project  schedules  for  the  delivery  of  capabilities  are  tight,  placing 
pressure  on  all  aspects  of  the  project  including  the  conduct  of  effective  T&E, 
particularly  the  critical  elements  such  as  the  testing  of  key  FIC  related  elements 
pertaining  to  the  measures  of  suitability.  Because  of  these  pressures,  there  is  a 
temptation  to  allow  inadequate  time  for  the  conduct  of  comprehensive  T&E  and 
the  need  for  regression  testing  as  a  result  of  rework. 

Schedule  slippages  impact  heavily  on  T&E  resource  availability  within  the  T&E 
agencies  and  these  slippages  often  lead  to  planning  issues  at  an  agency  level  and 
resource  clashes.  Resource  sequencing  issues  compound  project  delays,  usually 
resulting  in  less  T&E  being  conducted  than  was  planned. 

$ 

Currently,  contract  execution  compels  industry  to  focus  delivery  ‘on  time’  and 
‘on  budget’;  there  are  no  incentives  to  ensure  the  delivery  of  a  product  against 
the  capability  requirement  and  user  needs,  particularly  in  the  rapidly  changing 
technology  environment.  While  the  supplier  must  comply  with  contractual 
requirements,  there  could  also  be  performance  incentives  associated  with 
successful  completion  of  OT&E  activities  of  the  contracted  system. 

Governance 

It  must  be  noted  that  the  individual  T&E  agencies  with  limited  resourcing  have 
delivered  quality  T&E  that  has  enabled  capabilities  to  be  delivered  into  service  with 
minimum  risk.  However,  this  stovepiped  approach  will  not  be  adequate  to  support 
CDF’s  vision  for  an  ADF  that  is  a  ‘ balanced ,  networked,  and  deployable  force, 
...that  operates  within  a  culture  of  adaptability  and  excels  at  joint,  interagency,  and 
coalition  operations.  ’ 1 


Despite  a  continual  process  of  updating  T&E  policy  and  guidance,  there  are  still 
some  inconsistencies  in  policy  and  procedures  between  areas,  which  leads  to 
inconsistent  applications  of  T&E  across  Defence.  Steps  have  begun  to  address 
this  through  the  creation  of  the  ADTEO  and  its  mandate  to  provide  Defence  with  a 
more  strategic  and  cohesive  approach  to  T&E.  While  the  ADTEO  will  not  exercise 
command  or  technical  authority  over  other  T&E  organisations,  it  will  undertake 
management  and  coordination  at  a  joint  level  and  undertake  strategic  planning  to 
ensure  resources  are  available  for  the  other  T&E  organisations. 

In  the  past,  an  Acceptance  into  Operational  Service  (AIOS)  policy  has  been  largely 
absent,  and  as  a  consequence  AIOS  has  not  been  consistent  across  projects.  AIOS 
policy  documentation  has  now  been  developed  and  is  detailed  in  DI(G)  OPS  45-2, 
Capability  Acceptance  into  Operational  Service.  While  there  are  T&E  agencies 
directly  in  support  of  the  Capability  Managers  in  the  Maritime,  and  Air  and  Space 
environments  who  are  involved  in  the  AIOS  process,  there  is  no  dedicated  agency 
directly  supporting  this  aspect  for  the  Land  environment. 

1  Joint  Operations  for  the  21st  Century,  page  3 


Process  &  Systems 

It  would  be  more  efficient  if  Defence  adopts  a  model  where  T&E  subject  matter 
experts  are  engaged  during  the  concept  and  requirements  development  stage 
for  each  project  to  assist  with  the  specification  of  project  T&E  requirements. 
Identifying  problems  earlier  will  reduce  the  cost  of  ownership.  (Refer  to  Figure  3) 

The  ADTEO  has  begun  to  develop  processes  to  link  forecast  T&E  requirements  as 
described  by  project  Test  Concept  Documents  (TCD)  with  the  development  and 
maintenance  of  Defence  T&E  capability.  The  development  of  processes  needs  to: 

•  Encompass  both  Major  and  Minor  projects.  Presently  many  Minor  projects 
introduce  significant  Command  Control  Communications  Computers 
Intelligence  Surveillance  and  Reconnaissance  (C4ISR)  capability  or  weapons 
systems  upgrades  to  Defence,  but  are  not  generally  subject  to  joint  T&E  before 
fielding. 

•  Monitor  project  progress  to  accommodate  schedule  slippage,  scope  change, 
project  cancellation  and  short  notice  T&E  requirements  brought  on  by 
operational  need.  Many  Defence  projects  include  significant  developmental 
risk  that  make  long  term  T&E  forecasting  difficult. 

•  Assess  the  merits  of  developing  and  maintaining  a  particular  capability 
domestically  versus  making  use  of  international  facilities.  Consideration  will 
be  given  to  attracting  international  customers  to  Australian  based  T&E  facilities 
(where  there  are  no  national  security  issues). 


•  Consider  the  appropriate  level  of  centralisation  of  T&E  governance  and 

capability  development.  Many  elements  of  the  management  and  delivery  of  T&E 
capabilities  are  best  decentralised.  Environmental  based  T&E  organisations,  in 
particular  Maritime  and  Air  and  Space,  have  highly  developed  platform  level  T&E 
capabilities  that  should  be  supported  and  nurtured  by  Defence.  It  would  also 
make  sense  to  allocate  the  lead  responsibility  for  some  elements  of  joint  T&E 
to  the  existing  environmental  T&E  organisations.  However  it  is  apparent  that 
significant  elements  of  joint  T&E  are  not  presently  undertaken  by  Defence  and 
do  not  easily  exist  within  the  remit  of  any  particular  environmental  organisation 
-  these  should  be  managed  centrally.  As  NCW  concepts  are  embraced  by 
Defence,  the  joint  T&E  requirement  will  grow  to  rival,  or  even  overshadow,  the 
existing  environmental  based  T&E  requirement. 

$ 

Process  and  system  development  will  take  time,  but  is  crucial  to  development  of 
a  mature  Defence  wide  T&E  capability.  ADTEO  should  be  adequately  resourced 
and  supported  to  ensure  that  these  processes  and  systems  are  developed  and 
implemented,  to  enable  the  conduct  of  joint  T&E  with  multiple  platforms  and 
systems  in  the  future. 


CHAPTER  6  -  FUTURE  T&E  ISSUES 


The  CDF’s  vision  is  for  the  ADF  to  be: 

...a  balanced,  networked,  and  deployable  force,  staffed  by 
-0-  dedicated  and  professional  people,  that  operates  within  a 

culture  of  adaptability  and  excels  at  joint,  interagency,  and 
coalition  operations. 1 

In  order  to  achieve  CDF’s  vision,  Defence  has  developed  a  FJOC, 
which  provides  an  overarching  concept  for  the  ADF  to  be  supported 
by  concepts  describing  the  ADF’s  operation  in  the  three  major  combat 
environments:  Maritime,  Land,  and  Air  and  Space. 

The  FJOC  is  a  component  of  the  Strategic  Planning  Framework  and 
is  aligned  with  the  current  classified  Defence  Planning  Guidance  and 
the  Defence  Capability  Strategy,  which  remain  the  guiding  documents 
for  capability  developers.  Figure  6  describes  the  FJOC  as  part  of  the 
Defence  hierarchy  of  concepts. 


1  Joint  Operations  for  the  21st  Century,  page  3 


A  key  enabler  of  the  future  balanced,  joint,  deployable,  networked  force  is  NCW. 
Network  centricity  will  help  Defence  to  link  sensors,  engagement  systems,  and 
decision  makers  into  an  effective  and  responsive  whole. 

With  the  emphasis  on  joint  and  coalition  operations,  the  ADF  could  be  well  placed 
to  leverage  allied  systems  in  operations.  NCW  seeks  to  provide  the  future  force 
with  the  ability  to  generate  tempo,  precision  and  combat  power  through  shared 
situational  awareness,  clear  procedures,  and  the  information  connectivity  needed  to 
synchronise  our  actions  to  meet  the  commander’s  intent. 

As  noted  in  Joint  Operations  for  the  21st  Century,  Defence  may  not  always  be 
the  lead  Government  agency  for  dealing  with  security  challenges.  An  integrated, 
multi-agency  response  approach  will  increasingly  be  a  key  to  success  in  the  future 
strategic  environment,  and  so  Defence  will  routinely  work  more  closely  with  other 
Government  departments,  especially  in  area  such  as  C4ISR.  In  any  testing  of 
capability  that  involves  working  with  other  Government  departments,  the  ADTEO 
will  be  the  Joint  OT&E  Program  Manager. 

The  FJOC  will  drive  the  development  of  new  capabilities,  which  in  turn  will  drive 
systems  complexity.  The  complexity  of  these  new  systems  -  coupled  with 
cost  constraints  -  will  become  key  determinants  in  future  acquisition  policy  and 
approaches. 


Figure  6:  FJOC  Hierarchy  of  Concepts 


In  order  for  the  FJOC  to  be  realised, 
each  of  the  three  combat  environments 

-  Maritime,  Land,  and  Air  and  Space 

-  must  develop  and  deliver  into  service  the 
appropriate  capabilities  to  maximise  their 
contibution  to  mission  success  in  a  variety 
of  joint  or  coalition  operations. 

In  parallel  with  the  evolution  of  the  FJOC 
is  the  evolution  of  T&E  in  order  to  address 
the  complexities  and  risks  associated 
with  these  joint,  interagency  integrated 
operations. 

Accordingly,  the  remainder  of  this 
T&E  Roadmap  addresses  not  only  the 
measures  to  overcome  the  current  T&E 
issues,  but  also  the  conditions  which  need 
to  be  in  place  in  order  to  manage  and 
conduct  T&E  for  the  FJOC. 
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THE  FUTURE 


CHAPTER  7  -  FUTURE  APPROACHES 
TO  T&E 


Traditionally  in  an  ADF  heavily  weighted  toward  countering  a 
conventional  threat  against  Australian  territory,  T&E  has  mostly  been 
applied  at  the  platform  level  within  the  combat  operating  environments. 
As  the  ADF  moves  towards  a  hardened,  networked,  deployable  joint 
force  -  characterised  by  adaptability  and  agility  to  handle  the  full  range 
of  military  operations  across  the  full  spectrum  of  conflicts  -  there  will 
be  the  need  to  respond  to  these  technological  challenges  to  facilitate 
this  change.  Increasingly,  T&E  will  be  used  to  identify  integration  risks 
between: 

•  platforms  within  each  of  the  combat  operating  environments;  and 

•  the  Maritime,  Land,  and  Air  and  Space  combat  environments  in  the 
conduct  of  joint,  interagency  or  coalition  operations. 

Future  T&E  effort  in  Defence  aims  to  ensure  complex  systems  and 
capabilities  are  able  to  operate  across  each  of  the  three  combat 
environments  within  acceptable  risk  boundaries  to  achieve  the 
operational  requirements  of  the  mission. 
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THE  CHANGING  TECHNOLOGICAL  ENVIRONMENT 

T&E  requires  a  framework  which  will  enable  it  to  accommodate  the  complexity  required 

for  a  balanced,  networked  and  deployable  force. 

•  Revolutionary  and  Technological  Change.  This  is  occurring  on  an  exponentially 
growing  scale,  and  demands  that  Defence  institute  a  continuous  transformation 
process  to  include  all  areas  of  doctrine,  training,  leadership,  organisation,  and 
technology.  This  technological  change  has  two  characteristics.  First,  it  will  include 
synergy  of  emerging  technologies  such  as  bio-engineering,  nano-engineering, 
robotics  and  artificial  intelligence.  Second,  it  will  occur  at  a  speed  never  seen 
before. 

•  Software  Development.  A  more  systematic  and  rigorous  approach  to  software 
testing  is  required.  To  be  effective,  this  approach  must  be  applied  to  all  phases  of 
the  development  process  in  a  planned  and  coordinated  manner,  beginning  at  the 
earliest  design  stages  and  proceeding  through  to  OT&E  of  the  integrated  system. 
This  emphasis  is  driven  by  software  developments  of  weapon  systems,  such  as 
avionics  systems,  weapons  targeting  and  control,  and  navigation  computers. 
Standards  for  software  development  need  to  be  tailored  on  a  project-by-project 
basis  to  ensure  T&E  requirements  are  accommodated. 

•  Testing  of  Electronic  Combat  (EC)  and  C4ISR  Systems.  These  systems  pose 
unique  problems  for  testers  because  of  the  difficulty  in  measuring  their  operational 
performance.  Compatibility,  interoperability,  and  integration  are  key  performance 
areas  for  these  systems.  Special  testing  techniques  and  facilities  are  normally 
required  in  EC  and  C4ISR  testing,  such  as  shielded  electronic  chambers  and  threat 
generators,  with  a  potentially  heavy  reliance  on  M&S. 


•  Evolutionary  Acquisition  (EA).  Also  known  as  spiral  development,  EA  is  a 
strategy  designed  to  provide  an  early,  useful  capability  even  though  detailed 
overall  system  requirements  cannot  be  fully  defined  at  program  inception.  The 
EA  strategy  contributes  to  a  reduction  in  the  risks  involved  in  system  acquisition, 
since  the  system  is  developed  and  tested  in  manageable  increments.  The  C4I 
systems  are  likely  candidates  for  EA  because  they  are  characterised  by  system 
requirements  that  are  difficult  to  quantify  or  even  articulate  and  are  expected 
to  change  as  a  function  of  scenario,  mission,  theatre,  threat,  and  emerging 
technology. 

The  challenge  exists  to  ensure  that  T&E  resources,  systems  and  processes  are 
sufficient  to  deliver  capabilities  into  the  future.  Key  to  meeting  this  challenge  is  to 
ensure  that  T&E  personnel  are  adequately  educated,  trained  and  skilled  to  apply 
T&E  correctly  in  the  changing  technological  environment. 
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FACTORS  INFLUENCING  THE  WAY  FORWARD  FOR  DEFENCE  T&E 

As  Defence  moves  towards  a  complex,  joint,  NCW  environment,  a  number  of 
factors  will  impact  on  the  level  and  amount  of  T&E  required  to  ensure  appropriate 
linkages  exist  between  capabilities  for  joint  operations,  and  risk  is  contained  within 
acceptable  boundaries.  Factors  driving  the  shape  of  T&E  include: 

•  shortened  life  cycles  of  sub-systems,  systems  and  even  capabilities,  as  the  rate 
of  technology  innovation  continues  to  accelerate,  will  place  greater  burdens  on 
the  procurement  system  and  its  resources,  which  includes  the  conduct  of  T&E 
-  the  length  of  the  procurement  process  means  that  sub-systems  or  systems 
may  become  obsolete  prior  to  the  capability  being  introduced  into  service; 

•  the  increasing  degree  of  NCW  integration  at  all  levels  will  increase  the  amount 
of  practical  T&E  required  and  demand  an  ongoing  requirement  to  develop  new 

{§)  testing  methodologies; 

•  the  increasing  complexity  of  technology  -  involving  electronics,  information 
communications  technology  (ICT)  and  processing  power  of  components  will 
place  a  greater  reliance  on  software  intensive  systems; 


•  systems  and  solutions  acquired  to  meet  the  challenges  of  this  complex 
environment  will  have  a  higher  degree  of  systems  development  and 
technological  risk  -  increased  capability  complexity  will  increase  the  interaction 
and  reliance  on  sub-systems  and  systems  within  platforms,  which  means  that 
current  test  methods  are  likely  to  become  obsolete; 

•  the  disproportionate  effect  of  relatively  small  sub-systems  (particularly  in  ICT)  on 
major  platform  integration; 

•  changes  in  strategic  direction;  and 

•  emerging  asymmetric  threats  and  non-traditional  military  scenarios. 

Table  1  outlines  a  number  of  T&E  considerations  in  a  sample  of  non-traditional 

military  scenarios. 
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Emerging  Non- 
Traditional  Warfighting 
T&E  Concerns 

T&E  Objectives 

T&E  Issues  / 
Requirements 

Management  Information 
and  Communications 
Support  Systems 

Determine  vulnerability  of 
systems,  under  Electronic 

Warfare  (EW)  conditions. 

Determine  critical  nodes 
in  information  transfer 
process. 

Define  EW  threats  and 
engagement  concepts. 
Establish  equipment 
performance  threshold 
criteria  Identify  means 
to  intercept  information 
transfer  and  processing. 

Evaluate  means  to  enhance 
survivability  of  C4I  systems  by 
adapting  design  changes  and 
process  improvements 

Develop  test  data  for  V&V  of 
modelling  and  simulation 

Show  impact  of  design 
changes  /  hardening 
on  survivability 
improvements. 

Create  knowledge  base 
on  phenomenology 
associated  with  coupling 
of  electronic  effects  to 

C4I  equipment. 

Determine  sensitivity  to 
countermeasures  and 
engagement  constraints. 

Directed  Energy 

Weapons  (DEW)  Systems 

Determine  susceptibility  of 
combat  systems  to  damage 
from  DEW  systems 

Develop  data  to  support 
methodology  for  predicting 

DEW  effects  using  M&S  and  test 
data. 

Investigate  compatibility 
of  Electromagnetic  Pulse 
(EMP)  hardening  for  other 
DEW  systems  including 

RF  and  lasers. 

Expand  knowledge 
base  on  coupling  effects 
of  DEW  on  combat 
systems. 

Determine  sensitivity  to 
countermeasures  and 
engagement  constraints. 

Establish  bounds  on  lethality 
of  DEW  against  threat  combat 
systems. 

Develop  insights  into 
utility  of  focussed 
non-nuclear  EMP,  RF 
weapons,  and  lasers 
against  threat  combat 
systems. 

Identify  means  to 
enhance  performance 
of  DEW  against  threat 
systems. 

Emerging  Non- 
Traditional  Warfighting 
T&E  Concerns 


T&E  Objectives 


T&E  Issues  /  Requirements 


Determine  vulnerability  of 
emerging  combat  systems  to 
anti-armour  robotic  systems. 

Evaluate  implications  of  target 
size,  mobility,  and  stealth 
capability  to  achieve  target  kills. 

Robotic  Combat  Systems 

Assess  lethality  of  weapons 
against  threat  robotic  systems. 

Investigate  options  and 
techniques  for  defeating  robotic 
threat  systems  with  lethal  and 

DEW  systems. 

Counter  Terrorist 

Weapons 

Determine  capability  of 
systems  to  withstand  chemical/ 
biological  attacks. 

Translate  threat  scenarios  into 
framework  for  test  planning 
including  simulations,  burst 
locations  from  targets,  and 
instrumentation 

Determine  vulnerability  of  combat 
vehicle  occupants  as  a  function  of 
test  conditions  and  engagement 
parameters. 

Determine  capability  of 

ADF  personnel  to  withstand 
chemical/biological  weapons 
by  onboard  rapid  detection  and 
protective  measures. 

Assess  utility  of  current  and 
planned  measures  to  protect 
personnel  with  detection  devices 
and  inoculation,  protective 
clothing,  overpressure,  etc. 

Improvised  Explosive 
Devices  (IED) 

Determine  capability  of 
armoured  &  vehicle  systems  to 
withstand  IED  attacks  Translate 
threat  scenarios  into  framework 
for  test  planning  including 
simulations,  burst  locations  from 
targets,  and  instrumentation. 

Determine  vulnerability  of  combat 
vehicle  occupants  as  a  function  of 
test  conditions  and  engagement 
parameters. 

Determine  capability  of  ADF 
personnel  in  vehicles  to  defeat  IED 
attack  by  detection,  protection 
and  Tactics,  Techniques  & 
Procedures  (TTP)  measures. 

Assess  utility  of  current  and 
planned  measures  to  protect 
combat  vehicle  occupants  with 
detection  devices,  electronic/RF 
protection,  protective  clothing, 
safety  systems,  Automatic  Fire  & 
Explosion  Suppression  System 
(AFESS),  etc. 

Determine  effectiveness  of  system 
when  employed  utilising  current  or 
planned  TTPs. 

Table  1 :  T&E  Considerations  for  Non  Traditional  Military  Scenarios 


COMPREHENSIVE  T&E  THROUGHOUT  THE  SYSTEMS  ENGINEERING 
PROCESS 

There  is  a  need  -  based  on  the  approach  required  to  meet  the  complex,  joint, 
NCW  environment  capability  requirements  -  for  T&E  agencies  and  testers  to  have 
a  joint  T&E  and  platform  integration  focus,  regardless  of  traditional  organisational 
ties.  Operational  test  agencies,  Defence  laboratories  supporting  DT&E  ,  Industry 
design  engineers,  program  managers,  logisticians  and  users  need  to  cooperate 
in  order  to  achieve  cost-effective  solutions.This  cooperation  will  be  facilitated  at 
the  Defence  level  by  the  ADTEO,  and  at  the  system/platform  level  by  Capability 
Systems  and  DMO. 

T&E  expertise  must  be  included  during  program  conception  to  ensure  problems 
are  identified  and  addressed  early,  rather  than  identified  too  late  for  meaningful 
and  cost-effective  changes  to  be  made.  T&E  needs  to  provide  an  integrated 
continuum  of  supporting  activities  for  systems  engineering  verification  and 
operational  performance  exploration  in  realistic  threat  and  environmental 
conditions.  Systems  Engineering  and  T&E  Master  Plans  (SEMP  and  TEMP) 
need  to  be  aligned  with  Capability  Development  Documents  (CDD)  in  order  to 
articulate  this  integration.  The  focus  of  individual  tests,  testing  organisations,  and 
recipients  of  their  reports  may  be  different,  but  the  end  goals  should  align  towards 
expeditious  introduction  of  cost  effective  capabilities  to  the  warfighter.  Whenever 
feasible,  DT&E,  AT&E  and  OT&E  events  as  well  as  other  T&E  and  V&V  activities 
will  be  combined  to  make  optimal  use  of  resources.  The  operator  should  also  be 
involved  early  in  test  planning  to  ensure  the  capabilities  are  delivered  as  intended. 


POLICY  AND  GOVERNANCE 

The  ADTEO  is  reviewing  all  T&E  policies,  procedures  and  guidance  in  order  to 
develop  a  standardised  framework  which  ensures  consistency  and  alignment  at  all 
levels  across  the  operating  environments,  including  FIC  elements  and  cyberspace. 
The  Roadmap  does  not  seek  to  replicate  this  work.  Additionally,  this  review  will 
include  all  areas  that  undertake  T&E. 

An  overarching  T&E  strategy  flowing  from  the  DCP  should  be  developed  to  ensure 
future  T&E  resources  are  adequate  to  meet  the  DCP  and  the  capabilities  that  will 
operate  in  the  FJOC.  ADTEO  has  commenced  this  work,  and  is  best  placed  to 
coordinate  this  strategy  development. 

Specific  T&E  guidance  should  also  be  developed  to  provide  greater  assistance  in 
the  development  of  the  T&E  elements  of  contracts  for  MCE  projects,  as  well  as 
assisting  CDG  to  ensure  T&E  aspects  of  Major  projects  are  adequately  resourced. 


CAPABILITY  DEFINITION  PROCESS  REVISION 

Program  managers  are  being  tasked  to  provide  complex  capability  systems  to  the 
user  faster,  with  fewer  resources,  in  compliance  with  more  regulatory  and  statutory 
requirements. 

Legacy  acquisition  processes  bias  T&E  towards  design  verification  through 
segmented  contractor,  developmental,  live  fire,  and  OT.  With  increasingly  complex 
systems  and  greater  cost  constraints,  T&E  needs  to  transform  into  a  truly  integrated 
continuum  of  requirements  verification,  technology  maturation,  risk  management, 
capability  validation,  and  support  assessment.  This  continuum  should  itself  be 
interwoven  at  each  iteration  of  the  systems  engineering  process  throughout  the 
acquisition  life  cycle. 
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Accordingly,  the  capability  development  and  acquisition  processes  (which  include 
TCD  preparation)  require  revision  to  ensure  that  TCDs  both: 

•  evolve  as  a  project  progresses  (addressing  the  various  T&E  activities  to  be 
undertaken  during  the  life  cycle  stages);  and 

•  are  acted  upon,  to  ensure  that  all  necessary  resources  (including  financial)  are 
available  for  T&E  as  required. 

These  efforts  will  support  long-term  transformation,  with  a  net  effect  of  reducing 
total  ownership  costs  while  enabling  more  rapid  fielding  of  needed  capabilities 
through  intelligent  risk  management. 


RESOURCES 
Professional  T&E 

T&E  training  is  expensive  and  extensive;  Defence  should  strive  to  retain  its  T&E 
personnel  to  facilitate  successful  delivery  of  capability.  Members  of  the  T&E 
community  should  be  provided  incentives  to  remain  within  that  community. 
Motivation  and  incentives  may  need  to  be  tailored  for  individuals,  but  options 
could  include  career  paths  and  career  management. 

There  is  a  need  for  a  professional  T&E  association  alignment  to  provide  focus  and 
forum  for  those  involved  with  T&E  in  Defence.  Such  an  association  will  provide  the 
network  to  facilitate  discussion  along  with  distribution  of  T&E  practices,  principles 
and  the  review  of  the  advancing  technologies. 

Facilities 

An  overarching  Defence  T&E  facilities  plan  needs  to  be  developed  to  ensure  that 
both  existing  facilities  and  ranges  such  as  Woomera  and  the  Under  Water  Tracking 
Range  are  viable  facilities  into  the  future,  and  that  new  facilities,  such  as  an  EEH 
Test  Facility  -  which  will  be  needed  for  joint  T&E  -  are  established  and  equipped  in 
appropriate  timeframes.  The  EEH  Test  Facility  has  been  recommended  in 
ANAO  Audit  Report  No  30  2002-2003  -  ‘Defence  Ordnance  Safety  and 
Suitability  for  Service ’ 


Equipment 

Policy,  procedures  and  guidance  (as  necessary)  need  to  be  established  to  ensure 
T&E  equipment  is  calibrated  and  maintained  to  required  standards. 

Whether  as  part  of  the  overarching  T&E  Facilities  Plan  -  or  as  a  separate  stand¬ 
alone  document  -  planning  should  be  undertaken  to  ensure  the  appropriate 
equipment  is  available  when  needed  for  particular  testing  programs.  Such  planning 
should  consider  the  relative  merits  of  Defence-owned  and  operated  equipment, 
Industry-owned  and  Defence-operated  equipment,  or  wholly  Industry-owned  and 
operated  equipment. 

Spectrum 

The  NCW  Roadmap  highlights  that  ‘Radio  Frequency  (RF)  spectrum  is  both 
a  national  and  Defence  asset’.1  As  RF  spectrum  is  a  finite  resource,  T&E  RF 
requirements  are  becoming  increasingly  constrained  by  competing  requirements. 
These  competing  requirements  include  the  increase  in  bandwidth  requirements  of 
emerging  weapons  systems,  the  need  to  record  telemetry  from  modern  weapon 
systems  for  T&E  purposes,  as  well  as  the  encroachment  by  civil  and  commercial 
RF  needs.  Future  communications  planning  needs  to  allow  bandwidth  for  the 
transmission  of  T&E  data  during  operations  to  enable  the  conduct  of  T&E  to  support 
operational  activities. 

Defence  needs  to  adequately  articulate  its  spectrum  needs  to  CIOG,  so  that  the 
Australian  Communications  and  Media  Authority  (ACMA)  understands  Defence  T&E 
requirements. 


1  NCW  Roadmap  2007,  Section  8-2 


T&E  ORGANISATION 

In  order  to  best  coordinate  T&E  efforts,  the  ADTEO  will  be  the  central  Defence 

T&E  organisation  that  is  responsible  for  the  management  and  coordination  of  T&E 

functions  across  Defence  (and  through  the  various  stages  of  the  capability  life 

cycle).  The  ADTEO’s  responsiblities  include: 

•  The  accurate  forecasting  of  T&E  through  the  Preliminary  TCD  (PTCD)  and 
TCD;  monitoring  and  coordination  of  realistic  and  efficient  T&E  through  the 
Acquisition  phase;  and  scheduling  the  conduct  of  realistic  joint  OT&E  during 
the  In-service  phase. 

•  The  coordination  of  Defence  T&E  facilities  and  ranges,  to  ensure  that  the 
appropriate  and  required  T&E  capabilities  are  available  to  meet  future 
capability  testing  requirements.  Given  budgetary  constraints,  this  may  require 
oversight  by  a  steering  committee  to  agree  on  priorities. 

•  Maintenance  and  oversight  of  projects  where  T&E  is  conducted  across 
environmental  boundaries  or  of  a  joint  operations  nature. 

•  Through  the  TCD  process,  assigning  ‘lead  agency’  status  to  T&E  organisations 
where  the  project  crosses  environmental  boundaries. 

•  Assigning  responsibility  for  the  coordination  and  strategic  management  of 
Defence  T&E  personnel  resources,  to  establish  a  competency  framework  and 
ensure  testers  are  accredited. 

•  Oversight  of  T&E  training  across  Defence. 


A  coordinated  joint  T&E  approach  should  be  developed  for  emerging  joint 
capabilities.  This  can  be  facilitated  through  the  ADTEO.  Such  an  approach  would 
take  into  consideration  the  synergies  possible  through  coordination  of  the  agencies 
involved  in  evaluating  elements  of  these  emerging  capabilities.  In  light  of  this, 
existing  T&E  agencies  and  resources  should  be  reviewed  to  identify  gaps  and 
overlaps. 

PARTNERSHIPS 

Defence  should  look  to  establish  T&E  relationships  with  Industry,  allies  and  coalition 
partners  to  enable: 

•  the  use  of  T&E  results  from  external  organisations  to  satisfy  Defence  T&E 
requirements  (avoiding  duplication  in  testing); 

•  world’s  best  practices  to  be  studied;  and 

•  collaborative  and  reciprocal  use  of  scarce  facilities  and  ranges  to  avoid 
inefficient  duplication  where  possible. 

The  development  of  the  Woomera  Test  Facility  into  a  world  class  T&E  facility  will 
encourage  allied  and  coalition  partners  to  utilise  the  facilities  which  in  turn  will 
increase  international  partnerships,  allow  for  possible  sharing  of  T&E  data,  assist  in 
offsetting  the  costs  of  facility  maintenance  under  a  user  pays  model,  and  alleviate 
high  costs  in  expensive  overseas  testing. 


CHAPTER  8  -  CONCLUSION 


The  Defence  T&E  Roadmap  highlights  the  current  T&E  situation,  the 
challenges  facing  T&E  within  Defence,  and  the  actions  necessary  for 
Defence  to  ensure  that  it  develops  and  maintains  effective  T&E  to 
®  deliver  future  capabilities  into  service. 

A  coordinated  T&E  approach  to  delivering  capability  must  be  planned 
early  and  managed  throughout  the  capability  life  cycle  to  ensure  that 
capability  outcomes  are  achieved. 

To  meet  the  challenges  of  the  joint,  complex  technological  future 
Defence  T&E  must: 

•  Centralise  through  the  ADTEO  (on  behalf  of  CCDG)  the 
coordination  of  all  T&E  resources,  policy  and  processes  to  ensure 
the  most  efficient  and  effective  application  of  T&E. 

•  Support  projects  to  ensure  that  the  planning  and  resourcing  of  T&E 
activities  during  a  capability’s  life  cycle  are  adequately  addressed. 

•  Ensure  that  fielded  systems  are  able  to  operate  across  platforms 
and  environments  to  support  future  joint,  interagency  and  coalition 
operations. 


•  Develop  incentives  for  the  suppliers 
of  capability  to  deliver  products  that 
meet  both  the  capability  requirements 
and  user  needs,  rather  than  just  an  £on 
time’  and  £on  budget’  approach. 

•  Support  existing  T&E  agencies  and 
stakeholders  in  their  delivery  of  T&E 
by  the  provision  of  adequately  skilled 
T&E  professionals  who  are  trained 
and  equipped  to  meet  emerging 
technologies  in  future  capability 
systems. 

•  Centralise  and  coordinate  the 
management  of  T&E  facilities  and 
equipment  to  ensure  projects  in  the 
DCP  can  be  delivered  into  service  on 
time. 

Through  the  continual  delivery  of  quality 
T&E,  the  ADF  will  receive  systems  that 
meet  operational  requirements.This 
will  ensure  that  the  warfighter  is  best 
equipped  for  balanced,  networked,  and 
deployable  operations  across  the  full 
spectrum  of  conflicts. 


T&E  GLOSSARY 


Term 

Definition 

Acceptance  into 
Operational  Service 
(AIOS) 

The  process  by  which  the  Fundamental  Inputs  to  Capability 
(FIC)  comprising  a  Capability  System  are  proven  to  meet 
endorsed  capability  requirements,  usually  specified  in  an 
Operational  Concept  Document  (OCD),  and  assembled  such 
that  in  all  aspects  the  capability  has  been  realised  and  is 
acceptable  for  operational  service. 

Acquisition  Phase 

This  is  the  third  of  the  five-phase  Defence  capability  life 
cycle.  The  Acquisition  Phase  is  the  process  of  procuring  an 
appropriate  materiel  system  to  meet  the  identified  requirements 
while  achieving  the  best  value  for  money  over  the  life  of  the 
system. 

Capability  Baseline 

The  materiel  system  requirements  defined  by  the  Capability 
Definition  Documents. 

Capability  Definition 
Documents  (CDD) 

The  CDD  define  the  Capability  System  baseline,  and  comprise 
the  OCD,  Function  and  Performance  Specification  (FPS)  and  the 
Test  Concept  Document  (TCD). 

Capability  Manager 
(CM) 

The  role  of  the  Capability  Manager  is  to  raise,  train  and  sustain 
in-service  capabilities  through  coordination  of  the  FIC. 

Capability  System 

The  Capability  System  is  the  combination  of  the  eight 
Fundamental  Inputs  to  Capability  which  are  the  standardised 
elements  required  to  delivery  capability. 

Contractual  Delivery 

Contractual  Delivery  occurs  at  System  Acceptance  is  the 
physical  movement  of  supplies  from  the  contractor  to  Defence. 

Critical  Issues 

Critical  Issues  are  the  issues  deemed  critical  because  any  one, 
if  not  resolved,  could  lead  to  the  failure  of  a  materiel  system  to 
achieve  Operational  Release  (OR).  Critical  Issues  are  usually 
categorised  as  either  Critical  Operational  Issues  (COIs)  or 

Critical  Technical  Parameters  (CTPs).  COIs  are  the  operational 
effectiveness  and  operational  suitability  issues  that  must  be 
resolved  in  OT&E  in  order  to  determine  that  the  system  has  the 
capability  to  perform  its  mission(s).  CTPs  are  quantitative  and 
qualitative  test  measurements  of  technical  data  that  provide 
information  on  a  system’s  performance. 

Defence  Trial 

A  Defence  Trial  is  a  Trial  which  is  conducted  by  the  ADTEO  and 
is  performed  on  capabilities  and  systems  that  require  T&E  that  is 
independent  of  a  group  or  single  service. 

Defence  Trial 

Directive 

A  Defence  Trial  Directive  implements  a  Defence  Trial  and  is 
issued  by  the  Trial  Authority.  The  Directive  specifies  the  Trial’s 
aim  and  objectives,  program,  tasks,  division  of  responsibility 
of  participating  authorities/agencies,  safety,  administrative  and 
logistic  arrangements,  reporting  and  command  and  control 
requirements. 
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Defence  Trial  Report 

A  Defence  Trial  Report  is  produced  by  the  ADTEO  at  the 
conclusion  of  a  Trial,  and  is  distributed  in  accordance  with 
the  requirements  of  the  Trial  Sponsor.  -  note  we  use  big  T  for 
Defence  Trial ,  little  t  generic  trial. 

Design  Acceptance 

Design  acceptance  certifies  that  a  system  is  fit  for  service  and 
poses  no  hazard  to  personnel,  public  safety  or  the  environment. 

Experimentation 

The  application  of  the  experimental  method  to  the  solution  of 
complex  defence  capability  development  problems  potentially 
across  the  full  spectrum  of  conflict  types,  such  as  warfighting, 
peace-enforcement,  humanitarian  relief  and  peacekeeping. 

Final  Operational 
Capability  (FOC) 

FOC  is  the  point  in  time  at  which  the  final  subset  of  a  Capability 
System  that  can  be  operationally  employed  is  realised.  FOC  is 
a  capability  state  endorsed  at  project  approval  at  Second  Pass, 
and  reported  as  having  been  reached  by  the  CM. 

Fundamental  Inputs 
to  Capability  (FIC) 

The  standard  list  for  consideration  of  what  is  required  to 
generate  ‘capability’,  comprising  organisation,  personnel, 
collective  training,  major  systems,  supplies,  facilities,  support, 
and  command  and  management.  It  is  to  be  used  by  Defence 
agencies  at  all  levels  and  is  designed  to  ensure  that  all 
agencies  manage  and  report  capability,  using  a  common  set  of 
management  areas. 

Initial  Operational 
Capability  (IOC) 

IOC  is  the  point  in  time  at  which  the  first  subset  of  a  Capability 
System  that  can  be  operationally  employed  is  realised.  IOC  is 
a  capability  state  endorsed  at  project  approval  at  Second  Pass, 
and  reported  as  having  been  reached  by  the  CM. 

Initial  Operational 
Release  (IOR) 

IOR  is  the  milestone  at  which  the  CM  is  satisfied  that  the 
initial  operational  and  material  state  of  the  Capability  System 
-  including  any  deficiencies  in  the  FIC  -  are  such  that  it  is 
safe  to  proceed  into  a  period  of  OT&E  leading  to  an  endorsed 
capability  state. 

In-Service  Date  (ISD) 

ISD  is  the  point  in  time  that  symbolically  marks  the  beginning 
of  the  transition  of  a  Capability  System,  in  part  or  full,  from  the 
Acquisition  Phase  to  the  In-Service  Phase.  ISD  coincides  as 
closely  as  is  practicable  with  IOR. 

Latent  Defect 

A  deficiency  in  design,  materials  or  workmanship  not 
discoverable  by  reasonable  care  or  inspection  prior  to  System 
Acceptance  which  causes  a  software  failure,  or  a  failure  of 
supplies  other  than  software  which  falls  outside  the  incidence  of 
random  failures  to  be  expected  from  those  supplies. 
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Materiel  System 

A  subset  of  the  Capability  System  and  is  the  combination  of 
the  Mission  System  and  the  Support  System.  The  Materiel 
System  covers  those  aspects  of  the  FIC  that  are  provided  by  the 
acquirer. 

Measure  of  suitability 

A  parameter  that  describes  the  success  or  otherwise  of  a 
system  being  able  to  sustain  its  assigned  role. 

Measure  of 
performance 

A  specific  measure  of  a  system’s  capability  to  which  a  numeric 
value  can  be  assigned. 

Measure  of 
effectiveness 

A  parameter  that  describes  the  ability  in  which  a  system 
accomplishes  its  assigned  role. 

Mission  System 

That  element  of  capability  that  directly  performs  the  operational 
function,  which  includes  platforms  (eg  ships,  vehicles  or 
aircraft),  distributed  systems  (eg  communications  network),  and 
discrete  systems  that  integrate  into  other  mission  systems  (eg 
radar). 

Net  Personnel  and 
Operating  Costs 
(NPOC) 

NPOC  is  the  difference  between  future  and  current  mature 
operating  costs  associated  with  a  capability,  facility,  system 
or  specific  item  of  equipment.  It  reflects  the  net  difference 
between  the  cost  estimates  to  operate  a  new,  upgraded  or 
replacement  capability  offset  by  the  guidance  (DMFP  funding) 
available  to  operate  the  current  capability,  across  all  affected 
Groups. 

Operational 

Capability  (OC) 

OC  states  are  intermediate  capability  outcomes  at  the  Project 
level,  where  required,  between  IOC  and  FOC.  They  are 
identified  as  Operational  Capability  2,  3,  etc  (OC2,  OC3). 

Operational  Concept 
Document  (OCD) 

The  OCD  is  the  primary  reference  for  determining  fitness  for 
purpose  of  the  desired  capability  to  be  developed,  and  is  a 
complementary  document  to  the  Function  and  Performance 
Specification  (FPS)  and  the  Test  Concept  Document  (TCD) 
which  form  the  Capability  Definition  Documents  (CDD)  to  define 
the  Capability  System  baseline. 

Operational 

effectiveness 

The  ability  of  a  system  to  perform  its  intended  function  over 
its  intended  operational  spectrum,  in  the  expected  operational 
environment,  and  in  face  of  expected  threats  when  operated  by 
typical  operational  personnel. 

Operational  Release 
(OR) 

OR  is  the  acknowledgment  by  the  relevant  CM  that  a  Capability 
System  or  subset,  has  proven  effective  and  suitable  for  the 
intended  role  and  that  in  all  respects  is  ready  for  operational 
service. 
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Operational  suitability 

The  capacity  of  the  system,  when  operated  and  maintained 
by  typical  operational  personnel  in  expected  numbers,  at  the 
expected  level  of  competency,  to  be  reliable,  maintainable, 
available,  logistically  supportable,  compatible,  interoperable, 
safe  and  is  ergonomically  satisfactory. 

Operational  T&E 
(OT&E) 

T&E  conducted  under  realistic  operational  conditions  to 
demonstrate  that  a  capability  is  safe,  suitable  and  effective 
for  its  intended  role.  OT&E  is  conducting  with  representative 
users  of  the  system,  in  the  expected  operational  context,  for 
the  purpose  of  determining  its  operational  effectiveness  and 
suitability  to  carry  out  the  role  and  fulfill  the  operational  needs 
that  it  was  intended  to  satisfy. 

Originator 

The  agency  requiring  a  Defence  Trial. 

Rapid  Acquisition 
(RA) 

RA  is  a  method  of  quickly  overcoming  recognised  operational 
deficiencies  in  order  to  meet  immediate  security  challenges. 

Support  System 

The  Support  System  is  the  sum  of  the  existing  support 
infrastructure  and  the  additional  support  elements  being 
generated  under  the  contract  to  enable  the  Mission  System  to 
be  effectively  operated  and  supported  to  meet  its  operational 
requirements. 

System  Acceptance 

System  Acceptance  is  the  acknowledgment  by  the  DMO 
project  authority  that  an  acquired  materiel  system  complies  with 
contractual  and  single  service  requirements  and  is  ready  to  be 
transitioned  to  the  In-Service  Phase. 

System  Specification 

The  System  Specification  defines  the  validated  requirements  for 
the  system,  and  the  contractor  and  the  Commonwealth  use  it 
as  the  basis  for  common  understanding  of  the  system  technical 
requirements. 

Test  and  Evaluation 
(T&E) 

T&E  is  a  process  to  obtain  information  to  support  the  objective 
assessment  of  a  Capability  System  with  known  confidence, 
and  to  confirm  whether  or  not  a  risk  is  contained  within 
acceptable  boundaries  across  all  facets  of  a  system’s  life 
cycle.  The  individual  terms  are  defined  as:  a  test  is  an  activity 
in  which  a  scientific  method  is  used  to  obtain  quantitative  or 
qualitative  data  relating  to  the  safety,  performance,  functionality, 
contractual  compliance,  and  supportability  of  a  system;  and 
evaluation  is  analysis  of  test  results  to  determine  (verify)  or 
prove  (validate)  something. 
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T&E  Master  Plan 
(TEMP) 

The  TEMP  is  the  plan  for  traceability  between  T&E  activities 
and  the  endorsed  Critical  Issues,  to  further  ensure  that  only  the 
required  testing  is  undertaken.  Results  of  T&E  planned  in  the 
TEMP  are  used  to  provide  proof  that  new  or  upgraded  capability 
meets  its  baseline,  is  safe  and  fit  for  purpose  throughout  its  life 
cycle. 

Trial 

A  trial  is  defined  as  an  activity  to  establish  the  performance  and/ 
or  characteristics  of  an  equipment,  system  or  concept  against 
a  specification  or  approved  capability.  In  this  context,  a  trial 
usually  takes  the  form  of  a  planned  process  aimed  at  exercising 
the  trial  item  in  its  actual  or  simulated  environment  to  produce 
data  for  evaluation.  Information  derived  from  analysis  of  the 
data  is  then  presented  in  the  form  of  a  report  for  consideration 
in  the  higher  decision  making  process. 

Trial  Agency 

Any  organisation,  unit  or  laboratory  with  a  responsibility  for 
implementing  the  Defence  Trial  Directive  in  whole  or  in  part  is 
termed  a  Trial  Agency.  For  some  trials,  more  than  one  Trial 
Agency  may  be  involved.  The  Defence  Trial  Directive  will  clearly 
define  the  division  of  responsibilities. 

Trial  Authority 

The  Director  General  Test  &  Evaluation  (DGT&E)  is  the  Trial 
Authority  for  Defence  Trials. 

Trial  Manager 

The  Trial  Manager  is  the  Australian  Defence  T&E  Office  (ADTEO) 
staff  member,  named  in  the  Defence  Trial  Directive,  who  is 
responsible  for  management  of  the  Defence  Trial. 

Trial  Sponsor 

The  Trial  Sponsor  is  the  appointment  in  Defence  responsible  for 
initiating  and  funding  a  trial. 

Trial  Support  Agency 

Any  organisation,  unit  or  laboratory  with  a  responsibility  for 
supporting  a  Defence  Trial  is  termed  a  Trial  Support  Agency. 

For  some  trials,  more  than  one  Trial  Support  Agency  may  be 
involved.  The  Defence  Trial  Directive  will  clearly  define  the 
division  of  responsibilities. 

Validation 

Proof  through  evaluation  of  objective  evidence  that  the  specified 
intended  end  use  of  a  product  or  system  is  accomplished  in  an 
intended  environment. 

Verification 

Confirmation  by  examination  and  provision  of  objective 
evidence  that  specified  requirements  to  which  a  product  or 
service,  or  aggregation  of  products  and  services,  is  built,  coded, 
assembled  and  provided  have  been  fulfilled. 
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ACM  A 

Australian  Communications  and  Media  Authority 

ADF 

Australian  Defence  Force 

ADTEO 

Australian  Defence  Test  &  Evaluation  Office 

AFESS 

Automatic  Fire  &  Explosion  Suppression  System 

AIOS 

Acceptance  Into  Operational  Service 

AMAFTU 

Aircraft  Maintenance  and  Flight  Trials  Unit 

AOSG 

Aerospace  Operational  Support  Group 

AT 

Acceptance  Test 

AT&E 

Acceptance  Test  and  Evaluation 

C4ISR 

Command  Control  Communications  Computers  Intelligence 
Surveillance  and  Reconnaissance 

CCDG 

Chief  Capability  Development  Group 

CDD 

Capability  Development  Documents 

CDG 

Capability  Development  Group 

CDF 

Chief  of  the  Defence  Force 

CIOG 

Chief  Information  Officer  Group 

DCC 

Defence  Capability  Committee 

DCP 

Defence  Capability  Plan 

DCDM 

Defence  Capability  Development  Manual 

DEW 

Directed  Energy  Weapons 

DMO 

Defence  Materiel  Organisation 

DSG 

Defence  Support  Group 

DSTO 

Defence  Science  and  Technology  Organisation 

DT 

Development  Test 

DT&E 

Development  Test  and  Evaluation 

EA 

Evolutionary  Acquisition 

T&E  ROADMAP  ABBREVIATIONS 


Abbreviation 

Definition 

EC 

Electronic  Combat 

EEH 

Electro-Explosive  Hazard 

EMP 

Electromagnetic  Pulse 

EW 

Electronic  Warfare 

FIC 

Fundamental  Inputs  to  Capability 

FJOC 

Future  Joint  Operating  Concept 

FPS 

Function  and  Performance  Specification 

JE 

Joint  Experimentation 

JLC 

Joint  Logistic  Command 

ICT 

Information  Communications  Technology 

IED 

Improvised  Explosive  Devices 

M&S 

Modelling  and  Simulation 

MCE 

Major  Capital  Equipment 

NCW 

Network  Centric  Warfare 

OCD 

Operational  Concept  Document 

OT 

Operational  Test 

OT&E 

Operational  Test  and  Evaluation 

PTCD 

Preliminary  Test  Concept  Document 

RAAF 

Royal  Australian  Air  Force 

RF 

Radio  Frequency 

SEMP 

Systems  Engineering  Master  Plan 

SEP 

Systems  Engineering  Process 

TCD 

Test  Concept  Document 

TEMP 

Test  &  Evaluation  Master  Plan 

TTP 

Tactics,  Techniques  &  Procedures 

V&V 

Verification  &  Validation 

DEFENCE  T&E  AGENCIES,  RANGES 
AND  FACILITIES 

Defence  has  a  number  of  test  agencies,  ranges  and  facilities  that  are  utilised  in  the  delivery  of 
T&E,  and  are  tabulated  here  for  convenient  reference.  Defence  T&E  can  be  carried  out  in  Defence 
Training  Areas  (TAs)  depending  on  the  nature  of  the  T&E.  The  list  and  management  of  Defence 
TAs  is  detailed  in  DI(G)  ADMIN  59-1 ,  Management  of  Defence  Training  Areas.  Category  5  TAs  are 
T&E  ranges  which  are  dedicated  principally  to  the  testing  of  equipment,  equipment  systems  and 
ammunition.  All  TAs  are  to  be  declared  a  Defence  Practice  Area  if  the  T&E  activity  or  facility  poses 
a  risk  to  public  safety.  The  primary  Category  5  ranges  are  tabled  below. 


NAME 

OWNER 

T&E  Function 

Australian  Defence 
T&E  Office 
(ADTEO) 

CDG 

The  ADTEO  is  the  Defence  authority  for  T&E.  The 
ADTEO  provides  advice  and  consultancy  on  T&E 
issues,  and  services  for  managing  joint  and  single- 
Service  trials  in  support  of  T&E  activities. 

Aircraft  Systems  Test  Squadron:  responsible  for 
the  planning  and  conduct  of  T&E  of  RAAF  and 

Army  state  aircraft  and  associated  systems. 

Aircraft  Stores  Compatibility  Engineering  Agency: 
responsible  for  the  clearance  of  compatible  State 
aircraft  stores  configurations  associated  with 
the  safe  and  effective  carriage,  employment  and 
jettison  of  stores  from  all  ADF  State  aircraft. 

Aerospace 

Operational 

Support  Group 
(AOSG) 

RAAF 

Aerospace  Systems  Engineering  Squadron: 
responsible  for  engineering  Systems  Design  and 
Development. 

Joint  Electronic  Warfare  Operational  Support  Unit: 
responsible  for  Operational  Support  of  all  ADF 
aerospace  EW. 

Woomera  Test  Facility:  maintains  and  operates 
instrumentation  primarily  from  the  Woomera  Air 
Weapons  Range  for  the  collection  of  aerospace 
and  weapons  performance  data  during  T&E. 

Air  Movements 
Training  and 
Development  Unit 
(AMTDU) 

RAAF 

Undertake  trials  and  tests  for  new  air  movement 
and  aerial  delivery  equipment. 

Conduct  trials  of  Army  air  transportable  equipment 
and  RAAF  equipment  requiring  air  movement. 
Conduct  trials  and  recommend  standard 
procedures  and  methods  for  the  carriage  of 
internal  and  external  loads  by  helicopter. 
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Aircraft 

Maintenance  and 
Flight  Trials  Unit 
(AMAFTU) 

RAN  (Fleet 

Commander  through 
COMAUSNAVAIRGRP) 

Flight  testing  of  naval  aircraft  prior  to  and  following 
modification. 

Evaluation  of  the  suitability  of  new  or  modified 
equipment  for  service. 

Flight  testing  of  naval  aircraft  role  equipment. 
Determining  safety  limits  for  aircraft  landing  on  and 
taking  off  from  ships. 

DSTO  Intelligence, 
Surveillance  and 
Reconnaissance 
Division 

DSTO 

Conducts  research  and  development  in  airborne 
and  surface  radar  systems,  conducts  radar  and 
electromagnetic  measurements. 

DSTO  Weapons 
System  Division 

DSTO 

The  Systems  Simulation  Centre  carries  out  T&E 
of  military  and  other  equipment,  primarily  by  use 
of  hardware-in-the-loop  techniques,  in  which 
equipments  and  sub-systems  are  tested  in  real¬ 
time  simulations  of  their  operating  environments. 

DSTO  Maritime 
Platforms  Division 

DSTO 

MPD  manages  a  number  of  test  sites  at  which 
products  and  materials  can  be  tested  for  durability 
in  extreme  conditions.  Temperature,  humidity, 
ultraviolet  radiation,  organic  action,  salt  spray  and 
marine  immersion  are  only  a  few  of  the  hazards  to 
which  materials  can  be  subjected. 

DSTO  Air  Vehicles 
Division 

DSTO 

Aircraft  structural  testing  is  conducted  in  two  large 
laboratories.  Smaller  scale  metal  and  composite 
components  and  test  specimens  are  tested  in 
smaller  testing  facilities. 

Jervis  Bay  and 
Mobile  Telemetry 
Data  Acquisition 
Facilities 

RAN  (RANRAU) 

A  fixed  facility  at  Jervis  Bay  and  a  mobile  facility 
serve  as  ground  based  telemetry  receiving, 
processing,  analysis  and  display  systems  for  the 
testing  of  RAN  and  RAAF  missiles. 

Joint  Proof  & 
Experimental  Unit 
(JPEU) 

VCDF  (JLC) 

The  JPEU  is  responsible  for  conducting  dynamic, 
static  and  environmental  tests  and  trials  of 
components  or  complete  systems  of  explosive 
ordnance  (EO),  non-EO  and  other  items  to  support 
Defence  and  external  clients. 
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Land  Engineering 
Agency  (LEA) 

DMO 

LEA  has  extensive  facilities  and  equipment  for 
performance  measurement  and  environmental 
testing  of  Land  Materiel. 

Magnetic 

Measurement 

Range  (MMR) 

HMAS  STIRLING 

RAN 

The  MMR  provides  measurement,  data  processing 
and  analysis  of  magnetic  fields  associated  with 

Steel  Hulled  vessels  and  Submarines. 

Proof  and 
Experimental 
Establishment  Port 
Wakefield 

VCDF  (JLC) 

Long  range  testing  and  measurement  of  ballistic 
performance  of  large  calibre  weapon  systems  as 
well  as  environmental  and  non-destructive  testing 
of  explosive  ordnance. 

Proof  and 
Experimental 
Establishment 
(P&EE)  Graytown 

VCDF  (JLC) 

Offers  a  soft  recovery  technique  known  as  Vertical 
Recovery,  in  which  projectiles  are  fired  almost 
vertically  and  subsequently  recovered  from  the 
soil.  P&EE  Graytown’s  capabilities  include  the 

Small  Arms  Test  Centre  and  Munitions  Test  Centre 
at  Benalla  and  the  Small  Arms  Test  Centre  at 
Mullawa. 

Shallow  Water 
Sound  Range 

Jervis  Bay 

RAN  (RANRAU) 

Conducts  underwater  radiated  noise 
measurements  on  RAN  vessels. 

Sydney  Harbour 
Degaussing  Range 
Facility 

RAN 

Provides  measuring,  data  processing  and  analysis 
of  magnetic  fields  associated  with  steel-hulled 
surface  vessels,  submarines  and  Mine  Counter 
Measures  Vessels. 

Under  Water 
Tracking  Range 
(UTR)  HMAS 
STIRLING 

RAN 

The  UTR  is  used  for  real-time  measurement 
of  surface  and  underwater  vehicle  dynamic 
performance,  sensor  alignment  trials  and  weapon 
system  trials.  Additionally  the  UTR  enables  air, 
surface  and  sub-surface  participants  to  conduct 
Torpedo  Firing  Exercises  and  practices. 

Woomera  Test 
Facility  (WTF) 

RAAF  (AOSG) 

The  WTF  is  a  test  facility  administered  by  AOSG. 
WTF  activities  are  coordinated  by  WTF  planning 
staff  and  can,  at  any  time,  use  part  or  all  of  the 
Woomera  Prohibited  Area  (WPA). 

